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METHOD  OF  CALCULATING  VARIANCES  AND  COVARIANCES  FROM 
THE  FUNDAMENTAL  DEFINITION  OF  THESE  QUANTITIES 
AND  THE  LAW  FOR  THE  PROPAGATION  OF  ERRORS 


by 


B.  J 


Dalton—^ 


and  Robert  E 


*  .  2/ 

Bar leau— 


ABSTRACT 

This  report  gives  the  mathematically  exact  equation  for  cal¬ 
culating  all  variances  and  covariances  of  the  constants  evaluated 
which  is  applicable  to  any  function,  regardless  of  the  relationship 
between  the  observables  and  the  constants,  and  was  developed  from 
the  fundamental  definition  of  these  quantities  and  the  law  for  the 
propagation  of  errors. 

INTRODUCTION 

As  far  as  we  are  aware,  all  authors  and  all  programs  available 
for  calculating  variances  and  covariances  are  based  on  the  assump¬ 
tion  that  the  formulas  that  apply  to  linear  problems  are  applicable 
to  non-linear  problems,  once  the  linearized  normal  equations  are 

1/  Research  chemist,  Helium  Research  Center,  Bureau  of  Mines, 
Amarillo,  Texas. 

2 J  Supervisory  research  chemist,  project  leader,  Thermodynamics, 
Helium  Research  Center,  Bureau  of  Mines,  Amarillo,  Texas. 
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formed.  We  reject  this  assumption,  preferring  to  calculate  variances 

and  covariances  from  the  fundamental  definition  of  these  quantities 

3/ 

and  the  law  for  the  propagation  of  errors  (4,  6)—  . 


3/  Underlined  numbers  in  parentheses  refer  to  items  in  the  list  of 
references  at  the  end  of  this  report. 


The  present  method  in  use  for  evaluating  variances  and  covari¬ 
ances  of  the  parameters  is  therefore  only  an  approximation.  How 
good  this  approximation  is  can  only  be  decided  by  testing  it  against 
the  mathematically  exact  equation  to  be  given  in  this  report.  This 
exact  equation  is  applicable  to  any  function  regardless  of  the 
relationship  between  the  observables  and  the  constants. 


EVALUATION  OF  THE  CONSTANTS 


Let  us  consider  a  set  of  n  experimental  data  points:  Y.  ,  N  , 

i(o)’ 

til 

x.  ,  x„  ,  x~  ,  ...  (i=l,2,  ...  ,n)  ,  where  Y.,  x  is  the  i—  observed 

1 .  2 .  3 .  i(o) 

l  1  1 

dependent  variable  and  x.  (j=l,2,  ...  ,k)  are  the  i —  observed  inde- 

Ji 

pendent  variables.  We  define 


F  =  F(Y . ;  x.  ;  3  )  =  0 

K  i’  j.’  Miri 

J  l 


(1) 


In  our  treatment  of  this  problem,  we  assume  that  the  x.  's  are 

-1  i 

accurately  known  and  that  random  errors  occur  only  in  the  observed 

Y.'s,  and  that  they  are  normally  distributed. 

Now  because  of  random  errors  in  Yf/  .  ,  when  Y.  .  N  is  substituted 

i(o)  i(o) 

in  equation  (1) ,  F  will  not  reduce  exactly  to  zero.  Let  F^  be  the 
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value  of  F  when  Y.,  N  and  x,  are  substituted  in  equation  (1); 

1(0)  j.. 

thus  , 


Fi  =  F(Yi(o);  Xj.;  Pm) 


(2) 


Equation  (1)  can  be  solved  for  a  calculated  so  that  F  reduces 
to  zero;  thus, 


F  =  F(Y.,  .  x.  ;  p  )  =  0 

i(calc)  j  ’  ^m 


(3) 


where  are  the  independent  variables,  corresponding  to  the 
observed  x  '  s  evaluated  at  the  i—  data  point  and  |3m  (m=l,2,  ...  ,k) 


m 


are  the  k  constants. 


Now  AY.,  the  residual  of  Y. ,  .  ,  is  the  difference  between  the 
i  i(o) 

observed  and  calculated  values.  This  is  not  the  true  random  error 
of  Y.,  *  because  we  do  not  know  the  true  value  of  Y.,  ..  However, 
we  can  maximize  the  probability  that  the  AY^'s  are  equal  to  the 
true  random  errors  and  this  is  just  what  the  principle  of  least 
squares  does.  The  principle  of  least  squares  says  that  we  maximize 
the  probability  that  the  AY^'s  represent  the  true  random  errors  by 
minimizing  the  sum  of  the  weighted  squares  of  the  residuals.  Thus, 
we  should  minimize  the  quantity 


n 


R 


■  Iv,  <*V 

i=i  i<» 


(4) 


where  W, 


J.T  is  the  weight  to  be  given  Y.,  ..  If  the  Y^  .  '  s  all 

Yi(o)  l(o)  l(o) 

have  the  same  precision  index,  then  they  will  have  the  same  weight 


' 


t 


, 
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.  £ 
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and  W. 


Y. 


i(°) 

index,  then 


=  1.  If  the  Y#/  .'s  do  not  have  the  same  precision 

1(0) 


W 


Y. 


i(o) 


Y  Y 
i(o)Yi(o) 


(5) 


where  L  is  a  constant  and  S 
a  particular  problem,  it  may  be  necessary  to  assume  W 


„  „  is  the  variance  of  Y, ,  .  .  In 

Y .  /  >Y,,  v  i(o) 

i(o)  i(o) 


=  1  in 


i(o) 


the  beginning;  however,  if  this  is  done,  then  the  residuals  should  be 

examined  to  see  if  there  is  any  statistical  evidence  of  the  residuals 

squared  being  a  function  of  the  Any  assumption  as  to  the 

variance  being  a  function  of  Y,,  .  can  be  checked  by  examining  the 

r(o) 


residuals.  In  any  event, 


to  be  evaluated. 


i(o) 


is  not  a  function  of  the  parameters 


To  minimize  R  with  regard  to  the  various  constants,  we  form  the 
k  partial  derivative  equations  from  which  the  k  unknown  constants  can 
be  obtained.  Thus,  we  have  a  system  of  normal  equations  of  the  form 


(Hr) 


1  Xj. ,Yi(o) ,P2’ 


•  BAY.. 


’Iv.  “O 


LI  Y‘<«>  #1  '"j  »2. 

J  i 


=  0  (6.1) 


33 


2  Xj  .  ,Yi(o)  ,ei,P3’ 


n 


-3AY. 


2Iwy.,  .AYi(rf 


i=l 


i(o) 


2  Xj.’Yi(o)’ei’e3’ 


0  (6.2) 


■ 


t 
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n 


0  (6.k) 


31’*  "  ,3k-l 


If  the  functional  relationship  between  the  observables  and  the 
constants  is  a  linear  one,  then  the  solutions  of  the  normal  equations 
will  be  straightforward;  however,  if  the  functional  relationship 
between  these  quantities  is  a  non-linear  one,  then  it  will  be  neces¬ 
sary  to  solve  the  normal  equations  by  an  iterative  procedure.  Even 
though  it  may  be  necessary  to  solve  the  normal  equations  by  a  series 
of  approximations,  it  is  important  to  realize  that  the  best  values 
of  the  constants  are  determined  such  that  the  normal  equations  are 
satisfied  exactly. 

The  method  used  by  the  Helium  Research  Center  for  handling  non¬ 
linear  regression  problems  is  the  Newton-Raphson  method  (7.)  .  In 
this  method,  the  exact  normal  equations  are  formed  and  the  problem 
is  linearized  by  expanding  the  exact  normal  equations  in  a  Taylor's 
series  expansion,  retaining  the  first  two  terms.  For  a  more  detailed 
discussion  of  the  principles  involved  in  non-linear  regression  pro¬ 
blems,  we  refer  the  reader  to  our  previous  reports  on  this  subject 


(1,  2)  . 


The  linearized  normal  equations  are  of  the  form 


(7.1) 


(7.2) 
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akl^l  +  ak2A02  +  •••  +  akkA@k  “  \ 


(7.k) 


where 


n 


11 


-dAY.t  o' 


n 


S  AY.,o 


r-  /OLiJL.'iv-'  v-,  /o  ux.\ 

■  Iv.  ,y .  ♦Iv,.,«fcr‘) 


i=l 


i(o)  'd3l  x4  ,Y  ,  N  ,S0, 


(8.11) 


ji*  i(o)  ,p2  ’  ’  ’  ’  i=l 


i(o)  93  x  xj . ,Yi(o) ’^2 5 ’ ' 


a21  a12 


f,  /SAY.vO 

Iv..(se) 


/dAY.kO 

(»r) 


i=l  l(o)  d@1  xji,Yi(o) ’^2 ’ *  * '  ^  Xj . ,Yi(o) ,01’ 


n 


■  3  AY.  \  o 


+  I  Y,  YiCairB 


i(o)  ^YYx.  ,Y..  . 
i-l  j1’  i(o) 


(8.12) 


akl  alk 


,3AY  o 

zy,  asst) 


,3AY.no 

fer) 


i=l  i(o)  01  xj.,Yi(o)  ,P2’  *  *  '  Xj.,Yi(o)’^l 


n 


r  ,3  AY.  .0 

+  )  Wv  AY?(tt — r~  ) 

L  y  ,  .  Ass  .as./ 

i=l  Uo;  i  k  x  ,Y  (  ) 


(8.1k) 


n 

V 


22  iV,  Aap 


,3AY..o 

(air) 


1=1  P2  Xj.’Yi(o)>ei> 


+  L  WY.  .  AYiL  2  /  V  Q 
i=l  l(o)  d02  Xj. ,Yi(o) ,P1’ 


n 

V 


(8  22) 
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£  rdAY.x 

o 

^dAY.i  o 

L  wy. .  (sp,  J 

i=l  1(°)  2 

)  ( 

x .  , Y .  ,3i >  •  •  • 

J .  1,  \  i  d  iiafc .V 

J1  (o) 

^k  x.  ,Y  ,3 

i(o)  H1 

n 

,  B2AY.no 

+  V  W  AY  ( 

L  Y.  /  x  i' 
1=1  l(0) 

'SB2S0kA.  ,  Y .  .  . 

i(o) 

2 

o 


and 


M- 


1 


o 

X 


(9.1) 


n 


=  -I 


i=l 


W„  AY 
i  (o) 


Y,  , 


,ciAY.xo 
0 — *) 

2  Xj.’Yi(o)  ,ei’e3’ 


(9.2) 


and  Ap  , 
Equations 


n 


dAY.vo 


M*, 


=  -IwY.,xA4apr) 


(9.k) 


i=l 


'i(o) 


k  Xj.,Yi(o)  ,P1’*  ‘  *  ,Pk-l 


A32>  •  •  •  jA3^  are  corrections  to  the  undetermined  parameters 


(6.1),  (6.2),  ...,  (6.k)  result  from  expanding  AY_^, 


(8.2k) 


(8 . kk) 


: 


■ 


ii  oo  3 
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/SAY. 

\ 

i 

t— i 

on 

so 

x  Y 

3±  i(o) 

\ 

uk 

x.  ,Y  .  (  . 
3±  i(o) 

•  BAY.. 


■ior) 


2  Xji’Yi(o)  1  ’^3  ’ 
in  a  Taylor's  series  expansion  about 

the  approximate  solution,  (AY  )°,  retaining  only  the  first  two 
terms . 

The  method  to  be  used  for  obtaining  the  solutions  of  our 
linearized  normal  equations  involves  finding  the  inverse  matrix, 
which  we  now  proceed  to  do. 

Let  A  represent  the  matrix  of  the  coefficients  of  the  linear' 
ized  normal  equations 


A  = 


11 

a12  ’ • - 

3  Ik 

21 

a22  ' * ‘ 

a2k 

kl 

ak2  •** 

akk 

or 


A  [  a  ,  ]  / 1  i  \  [^*]/ii\ 

jm  (k,k)  mj  (k,k) 


=  A 


(10) 


where 


,SAY.no 

V.Aair)  Y  fl  laTV  Y  fl  +ZWY..  y 

.  i(o)  j  x.  ,Y .  .  N  ,p  m  x.  ,Y.,  v  ,  p  .  /  .  ,  i(o)  m  x.  . 

*  1  J.  i(o)  mfj  J.  i(o)  jrm  1=1  J  j,  i(o) 


SAY.n  o 
i 


n 

+  )  W, 


d  AY.N  o 

AY°f 


Here,  j  ranges  from  1  to  k  and  denotes  the  row,  while  m  ranges  from 


1  to  k  and  denotes  the  column  of  the  element  a .  ,  and  A  of  order 


(U) 


* 

'  .  r 


‘ 


; 
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(k,k) ,  obtained  by  interchanging  rows  and  columns  in  matrix  A  of 
order  (k,k) ,  is  referred  to  as  the  transpose  of  A. 

Now  let  us  write  the  linearized  normal  equations  as  the 
single  matrix  equation 


[ajJ(k,k)tiPm](k,l) 


(k,l) 


(12) 


where  [AS  ]  ✓,  is  a  column  matrix 
m  (k  ,  1) 

a  column  matrix  of  k  rows. 

Let  b.  represent  the  elements 
nated  as  A  \ 


of  k  rows , 


and 


is 


of  the  inverse  matrix,  desig- 


A 


-1 


b - 

b 

b 

ii 

12 

ik 

b 

b 

...  b 

21 

22 

2k 

b 

b 

,  .  b 

kl 

k2 

kk 

or 


-i 


^bjiJ  (k,k) 


^bmj  (k,k) 


-1  T 

=  (A  L) 


(13) 


Now  A  is  just  that  matrix  which  when  multiplied  by  A  gives  the 
identity  matrix,  I 


10  ...  0 

0  1  .  .  .  0 

00  ...  1 


(14) 


(k,k) 


. 


i 


12 


and  we  see  from  equation  (14)  that  the  identity  matrix  ...  "is 
uniquely  defined  by  the  property  that  it  behaves  like  the  scalar 
1  in  matrix  multiplication"  (5).  Therefore,  multiplying  both  sides 
of  equation  (12)  by  A  \  we  get 

tABm](k,l)  =  [b  jm]  (k  ,k)  ^  (k ,  1) 

^mjhk.lO^jkk.l)  O-5) 

from  which  we  can  evaluate  the  A8  's,  where  the  AS  ' s  (m=l,  2,  . . . ,  k) 

m  m 

are  defined  as 

=  Sx  -  @1  (16-D 

Ap2  =  e2  -  (16.2) 

APk  =  Bk  '  Pk  (16’k) 

and  02  >  .  .  .  ,  3^  are  t*ie  un<^et:erm^ne<^  constants  while  3°,  3°> 

...,  3°  are  approximate  values  for  these  quantities. 

If  the  assumed  values  for  the  undetermined  constants  are  not 
too  good,  then  it  will  be  necessary  to  repeat  the  calculations 
using  the  new  3^>  32’  •••»  3^  as  the  start  of  a  new  iteration.  We 
continue  this  iterative  technique,  provided  the  problem  is  con¬ 
tinuing  to  converge,  until  jjl  =  ^  =  .  .  .  =  p,  =  0,  to  within  some 

predetermined  small  quantity,  the  final  3 ^ 5  ^2’  '*•’  being  the 
least  squares  solution  for  these  quantities.  For  a  more  detailed 
discussion  of  methods  to  use  that  will  lead  to  convergence  of  the 


■  1  to 

.  ■  ••  :  •  .  .  .  Up  .  . 

■,  .  •  ,  •  . 

.. 
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iteration  when  the  problem  starts  diverging,  we  refer  the  reader 
to  our  previous  report  (_3)  on  this  subject. 


CALCULATION  OF  VARIANCES  AND  COVARIANCES 
Once  we  have  determined  the  best  values  of  3-^,  .  .., 

we  proceed  to  calculate  all  variances  and  covariances  of  these 
constants.  We  do  this  from  the  fundamental  definition  of  these 
quantities  and  the  law  for  the  propagation  of  errors  (4 ,  6} . 

This  law  says  that  if  we  have  a  function  or  quantity,  say  Q, 
that  is  a  function  of  the  independently-observed  quantities  y^, 
y  ,  ...,  y  ,  then  the  variance  of  Q  is  given  as 


-Xfe*r) 


QQ 


dy,. 


i(o)'  yi(o)yi(o) 


(17) 


i=l 


2  2 
where  S  is  the  variance  of  y.,  .  ,  and  S^  is  the  variance 

yi(o)yi(o)  l(0)  QQ 

of  Q.  Extracting  the  square  root  of  the  variance,  we  obtain  a  value 

on  the  same  scale  as  the  function  Q.  This  value  is  referred  to  as 

the  standard  error  or  standard  deviation  of  Q. 

The  values  of  our  constants  which  we  have  evaluated  are  func¬ 


tions  of  all  of  the  observed  Y.’s  and  x.  's.  Since  we  assumed  the 

1  h 

errors  of  the  x.  ' s  to  be  zero,  then  the  expression  for  calculating 

Ji 

the  variances  of  the  constants  is  of  the  form 


Sg  0=1,  2,  k) 

pnrm 


(18) 


- 

t 

■ 


.  . 


v  •  ■  1 

T  i  .  •  ,  f  '  ••  ’  ■ 
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The  expression  for  calculating  the  covariances  of  the  constants 
is  of  the  form 


n 


2>  •••  k;  =  I  (a¥Tr)(sYTL) 


$3  , 


3  3 

mj 


i=l 


i(o) 


is;  y  (19) 

'i(o)'  i(°)  i(o) 


S3 


Now  in  order  to  evaluate  equation  (18)  ,  we  must  evaluate 


— )(m=l,  2,  ...  k)  ,  multiply  by  Sy 


i(o) 


i(o)Yi(o) 


square  the 


product  of  these  two  quantities  and  then  sum  the  resulting  pro¬ 
duct  over  all  of  the  observed  Y.'s.  Similarly,  in  order  to  evak 

1  /  aem  \ 

uate  equation  (19),  we  need  to  evaluate  - y(m=l,  2,  ...  k) 


S3, 


i  (°) 


se 


and  ^~y- ^ — ^(j  =  l,  ...,  k;  j^m) ,  multiply  the  product  of  the 

i(o)  2 

two  quantities  by  S  ,  then  sum  the  resulting  product 

i(o)  i(o) 

over  all  of  the  Y^^'s.  We  now  proceed  to  evaluate  these  quan¬ 
tities  . 

We  have  a  total  of  n  pairs  of  the  observed  quantities  Y^, 
x.  .  The  constants  are  determined  from  the  solutions  of  equations 

h 

(6.1),  (6.2),  ...,  (6.k).  Now  suppose  we  change  one  of  the  observed 

Y.'s,  say  Y_ ,  N  ,  to  [Y0/  N  +  AY„ ,  sl.  Then  on  solving  equations 
l  y  2(o)  2(o)  2(o)  °  ^ 

(6.1),  (6.2),  ...,  (6.k),  we  will  get  new  values  of  3-^?  32’  •••>  3^ • 
Then  we  can  calculate 


(se-)  ■ 


A3 


m 


2  (o) 


AY 


2(o) 


(m=l,  2, 


k) 


(20) 


This  means  that  when  we  change  Y~,  N  by  the  small  amount  AY„ ,  N  , 

2  (o)  y  2  (o) 

then  equations  (6.1),  (6.2),  ...,  (6.k)  must  still  hold  exactly. 


J  \ 
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Mathematically,  this  means  that 


(!B 


by.,  .  ; 

1(0)  mX.^Y1(o) 


0  (m= 1 ,  2  ,  . . . , k) 


(21) 


Now  the 


(g-) 


's  are  functions 


m  x .  ,Y. t  x 

i(o) 


and,  through  the  constants  evaluated, 


Y. ,  's.  Then,  under  these  conditions 

i(o) 


of  all  of  the  constants 

are  functions  of  all  of  the 
,  we  have 


(22.1) 


(22.2) 


...  ■  !  •  .. 


' 


" 


'  ■  "  .  ■  :  1  -  .vri  ■ 


.. 


- 


.  . 


Now  we  can  solve  equations  (22.1),  (22.2),  ...,  (22. k)  simul- 

(  ^31  \  /  ^2  '\ 
taneously  and  evaluate  the  derivatives  - J  ,  - J 


'  ’  VdY. ,  J 


1(0)  Yr#.  i(o)  Y^. 


Then  once  we  have  expressions  for  each  of  these 


l(0)  Vi  ,  s 

quantities,  we  can  multiply  each  derivative,  say  — - j  ,  by 

i (o)  Y 

rfi 

Sy  ,  square  the  product,  then  sum  this  product  over  all  of 

i(o)Yi<o) 

the  observed  Y^'s.  This  will  give  us  an  expression  from  which  we 
can  calculate  the  variance  of  |3  .  In  a  like  manner,  we  can  calculate 
the  variance  of  of  3^,  .  and  of  3^. 

In  order  to  evaluate  the  variances  of  our  constants,  we  see 


from  equations  (22.1),  (22.2),  ...,  (22. k)  that  we  need  expressions 


for 


\SB.3B  / 


(-th.  ) 

5  VdB.dB  J 


(  32r  ) 

’  WdB  J 


.33,^3  )  v  ’  \B39c>3  -  v 

1  m  x.  ,Y .  ,  N  2  m  x.  ,Y#/  N 

3±  i(o)  3±  !(°) 


S3kS3m'x.  , Y .  ,  v 
3t  1(°) 


and 


for 


(ft:) 


LdYi(o)  VS3i'x.  ,Y.f  v39,...jx.  ,Y  ,.,3t,. 

3±  i(o)  2  3±  r^i  K1 


r  d 

/dR  \ 

Lby..  . 

l(o) 

W 

r  d 

/dR 

LdY . (  s 

i(o) 

w  J 

‘j  .  ’  i(o) ’pl’p3 

Ji 


j t’  i(°) ,P1 


M  ’  r/i’pl 
J  i 


k- 1  i .  rfi  1 

l 


.  We  now  proceed 


' 


.. 


t  , 

- 
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to  evaluate  these  quantities. 

We  start  with  equations  (6.1),  (6.2),  ...,  (6.k): 


(Sr) 


n 


1  Xj.’Yi(o) ,P2’ *  * ‘ 


BAY.' 


/UUi. 

■ife4) 


i=l  l(o)  1  j . ,Yi(o) ,P2’ ' ' ' 


(6.1) 


(fir) 


n 


2  Xj.’Yi(o)’Pl’@3 


SAY.' 


:i(aer) 


2Zwy  y  flfl 

i=l  l(o)  2  Xji,Yi(o) ,P1,P3* ' * • 


(6.2) 


(Hr) 


n 


k  Xj.’Yi(o)’Pl’@2’ 


SAY 


=  2  )  W„  AY 


i=l 


Y .  ,  “*i\Sp 

i(o) 


(6 .  k) 


k  xj.’Yi(o)>ere2’- 


Let  us  differentiate  equation  (6.1)  with  regard  to  each  constant; 
thus , 


s2r 


-) 


S(3,S(3  <-  v 

1  Km  x .  ,Y. .  . 

3±  U°) 


n 


^  ,SAY..  /SAY.. 

2  Awy..  AST-)  Y  .  (aTY  Y 

1=1  l(o)  1  j.,Yi(o)’P2’"'  m  xj.’Yi(o) 


n 

+  2IW 


/,S2Ayi  ^ 

i\S6 . SB  / 


AY  , 

Y .  ,  .  i\S3,S3  .  v 

.  ,  i(o)  1  m  x .  ,Y.  .  . 

i=l  2±  i(o) 


(m=l ,2 , . . . ,k)  (23) 


;  1  9 


. 


And 


differentiating  equation  (6.1)  with  regard  to  a  single  Y^^ 


holding  all  constants  and  all  x.  '  s  fixed,  we  get: 

J  i 


i(o)  1  Xj  .  >^2  5 '  *  ’  x j  . ’^"r^i ’^1  ’  ’  ’  * 


(24) 


2W 


,dAY 


i  \ 


y^.ASYw.J 


(S 


f(°)  i(°)  x  j  >  Yr^  ^  >  •  •  •  ^  xj  ,  »^i(0)  *  ’  ’  ’ 


+2WV  AY . 
Y.  .  v  l 

i(o) 


© 


i  (o)  ^  xj  .  ,Yi(o)  ,P2’*  *  *  "j.’^i’^l 


x,  ,y  ,3,, 


Now  AY.,  the  residual  of  Y. ,  is  iust  the  difference  between  Y..  . 

i  i(o)’  J  i(o) 

and  •  Therefore, 


•  dAY. 


far1-) 


=  1 


(25) 


i ( o)  Xj  , Y^^ , 3  ^  •  •  • 


and 


dAY. 


1  l(0)  Xj.,Yr^i’Pl’ - "  Xj . ,Yi(o) ,P29, ' * 


.  /dAY. 
d  (  i 


.BY.,  .\d3.  /  V  Q 
i(°)  i  xji’Yi(0) »P2* ’ * 


.  x .  ,Y  ,  .  ,3  , . .  . 
j .  rf l  1 

l 


(26) 


Substituting  equations  (25)  and  (26)  into  equation  (24) ,  we  get 


.  (,v  ii  I  :  I  •  '  J  £>  ■'  >J  ■  f! 

(  ' 

’  '  A7i 


IP 

ir 


w 


I 


1 " )  ’ 


J. 
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Therefore,  we  have  from  equations  (23),  (27),  (21),  and  (22.1): 


11 


\SY. . 


+  a. 


i(o)  Y  ,. 

rf  l 


33, 


12\dY. 


,  d3 


+  ...  +  a 


i(o)  Y 


VaY.,  J, 


r^i 


lk\dY . (  1 

l(o)  Yr*i 


=  M,  (28.1) 


where 


n 


11 


r  ,dAY.N2 

=  1  WY  AdT")  Y 

l(0)  1  Xj.’Yi(o)’B2’- 


n 


I 


+  )  W„  AY 


Y. 


i=l 


i=l 


i(o)  •  d3 


(29 


1 


x  ,Y  ,3  , 

J  i  i(o)  z 


and 


,3 AY.\ 

M!  =  “wy  AapT") 

i(o)  1  X  ,Y  ,3  ,  . 

Ji  i(o)  z 


(30.1) 


Now  differentiating  equations  (6.2),  ...,  (6.k)  with  regard 
to  each  constant  and  then  with  regard  to  a  single  Y.^^,  we  get 
expressions  similar  to  equations  (23)  and  (27).  Then  upon  substi¬ 
tuting  these  expressions  into  equations  (22.2),  ...,  (22 .k),  recalling 


the  relationship  of  equation  (21),  we  get  expressions  similar  to 


.11) 


(29.12) 

’Yi(o) 


(29.1k) 

i(o) 


' 


■ 
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equation  (28.1)  of  the  form 


(  3pl  ) 

a"l\dY. /  J 


+  a. 


/  3@k  \ 

+  •  •  •  +  a?i.(^ - )  =  Mo 

i(o)'Yr)4i  “~‘l(o)'Yr^  2V3i(o)Yriti 


21\SY.  ,  J„  '  ~22\SY.  ,  J 


(28.2) 


33 


akl(dY.  .  J  +  ak2^Y  .)  + 

i(o)  Y^.  i(o)  Yrjt. 


33 


&), 


33i 


/  U|Jk  \ 

•  +  akk(s7— “  “k  (28A) 


1<°)  Yr*i 


where 


a21  al2 


(29.12) 


akl  alk 


(29.1k) 


n 


SAY.,  2 


a22=IWY.  (sT1)  +ZWY.  AYi. 

i=l  l(o)  2  Xj.,Yi(o)’^l,P3’,‘*  1=1  l(o)  SP2  Xj.’Yi(o) ’^l’^3’’ ’ 


n 


I 


/32AY., 

■itr1) 


(29.22) 


ak2  a2k 


n 


W. 


\ae„  J 


1=1  Yi(o)KSe2  'xji.Y1(o) ,B1,b3 


/3AY.* 

(air-1) 


32AY. 

V 

1=1  Ji  i(o) 


+i« 


k  •*!(«,)  >*1 


(29.2k) 


n 


/3AY  .N  2 

ai-1; 


n 

+  Y  W 


AY. 


Y .  ,  “*iV^n2 


2 

3  AY. 
i 


akk  L  WY.  ,  A33  r  v  q  q  '  v 

i=l  l(o)  k  Xj.,Yi(o) ,P1’ ‘ * • ,Pk-l  i=l  l(o)  dpk  Xj.’Yi(o) 


(29. kk) 


■ 
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and 


/dAY. 

M2  =  "WY 

i(o)  2  xj.>Yi(c)’ere3’ 


(30.2) 


/dAY-\ 

Mk  =  ~^Y-(  '  Y  R 

l(0)  k  Xj.,Yi(o),Pi,**',Pk-l 

l 


(30. k) 


Equations  (28.1),  (28.2),  ...,  (28. k)  can  be  written  as  the 
single  matrix  equation 


ry  30  x 


=  [MJ 


i(o)  Y  (k,l) 

rn 


j  (k,l) 


(31) 


where 


A  =  [a  .  ]  n  .  v  =  [a  .  ]  .  ,  =  A 

jm  (k,k)  mj  (k,k) 


(32) 


and 


£  .BAY.,  /BAY  .  £  /B2AY.  « 

3  jm  =  A  WY.  (  Aap.  /  ft  )  +Z  WY.  (  ,AYi\B3.B3  /  Y 

i=l  1(0)  J  Xji,Yi(o)  m  Xj.’Yi(o)’Vm  i-1  l(°}  J  “V  i<°> 


Here,  j  ranges  from  1  to  k  and  denotes  the  row,  while  m  ranges  from 

r /  \  ~ 

1  to  k  and  denotes  the  column  of  the  element  a.  ;  (— - ) 

jm  L\oY .  ,  ,/  J,.  -j , 

i(o)  Y,.,  (k,l) 

is  a  column  matrix  of  k  rows;  and  [M,]  ,  ..  is  a  column  matrix  of 

J  \k ,  L) 

k  r  ows . 


We  see  that  the  a^  element  defined  by  equation  (33)  is  of 
the  same  form  as  that  defined  by  equation  (11) .  The  difference 


(33) 


. 


. 


-  .  .  j  . 


. 


i  '  .  .  .  -  .  '•  "  ,  -  r;  •  •:;■.[  O  ■■ 

, 

.  .  ..  ‘  •  '  .  -1  it-  •'  ■■  I  '  •  •  -  • 
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between  these  two  definitions  is  that  equation  (33)  applies  when 


the  least  squares  solution  is  obtained  while  equation  (11)  applies 

to  a  trial  solution.  For  the  final  iteration,  a.  as  defined  by 

Jm 

equation  (33)  can  be  considered  as  the  value  of  the  a .  element  of 


the  coefficient  matrix 


Multiplying  equation  (31)  by  the  inverse  matrix,  A  ^ ,  we  get 


rv  S3 


&). 


j 


LVSYi(o)'Y  J(k,l) 
r^i 


=  A'1[Mj](k.D 


(34) 


where 


A’1  =  [b.  ],,  ..  =  [b  .  =  (A_1)T 

jm  (k,k)  mj  (k,k) 


(33) 


j  ranging  from  1  to  k  and  identifies  the  row,  while  m  ranges  from 

1  to  k  and  denotes  the  column  of  the  b.  element.  The  b.  element 

jm  jm 

defined  by  equation  (35)  is  of  the  same  form  as  that  defined  by 
equation  (13)  .  The  difference  between  these  two  definitions  is 
that  equation  (35)  applies  when  the  least  squares  solution  is  ob¬ 
tained,  whereas  equation  (13)  applies  to  a  trial  solution.  For 

the  final  iteration,  b.  of  equation  (35)  can  be  considered  as  the 

jm 

value  of  the  b.  element  of  the  inverse  matrix. 

Jm 

Therefore,  from  equations  (34)  and  (35),  we  have 


r/  S3 


1 


(  Hm  \ 

,.j(k,D  =  [V(k>k)[V(k,i) 

rfi 


(36) 


S3 


and  the  derivative 


e-) 


u°)  Vi 


is  of  the  form: 


. 


.  : i 


'  •  ' 
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f— I_\ 

^Yi(o)  Y  hllMl  +  b21M2  +  +  bklKk 

'  '  r^i 

=  bllMl  +  b12M2  +  •  '  •  +  blA 


(37) 


Now  multiplying  equation  (37)  by  S  and  then  squaring 

i(o)Yi(o) 

this  product,  we  have 


*>1  \2 
BY.  ,  x  / 


Y. 


i(o)  i(o)  i(o) 


2  2  2  2  2  2 
bUMl  +  b12M2  +  •  •  •  +  blkMk 


+  b  b  MM  +  b  b  MM 
11  12  1  2  12  11  12 


+  .  .  .  +  bnblkMlMk  +  blkbuM1Mk 
+  b12blk^2^k  +  blkb12M2Mk 


+ 


Y  Y 
i(o)*i(o) 


But  from  equation  (5) 


Yi(o) Yi(o) 


W. 


Y. 


i(o) 


(39) 


and  multiplying  equation  (39)  by  M^,  where  M^  is  defined  by  equa¬ 
tion  (30 . 1)  ,  we  get 


2  2 
M1  S 

i(o)  i(o) 


2 

L  W, 


,BAY.X2 


i(o) \BB , 


1  Xj.’Yi(o)’P2’ 


(38) 


(40) 


/ 


■ 


. 
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and,  in  general, 


M  M 
m 


cj2  /ml,  2  ,  , k\ 

j  Yi(o)Yi(o)^  =  1’2,  "  ' 


9  ,dAY.x  ,SAY  .* 

=  L\(o)^x  Y  6  Mx  Y  B  (41) 

i(o)’Kj£n  J  Ji>  i(o)  Vj 


Substituting  equation  (41)  into  equation  (38)  and  then  summing  over 

all  of  the  Y . /  * ' s ,  we  get 

i(o) 


--t-)s2  =  S2 

BY.  ,  x /  Y .  ,  J. ,  N  6,8, 
i(o)  i(o)  l ( o)  M1M1 


k  k 

^  I 

m=l  j=l 


n 


SAY.. 


i  IHY.,  Xw1) 


blmblj  ZA 


/SAY.v 

(sr1) 


.  .  i  (o)  Km  x.  ,Y.  /  *,p.,  x.  ,  Y .  .  ,  ,3  ,  . 

i=l  Ji  i(o)  ,Kj?hn  J  ji’  i(o)  m/j_ 


which  is  the  mathematically  exact  equation  for  calculating  the  variance 


of  3^ 


Now  the  derivative 


( 


33 


2\ 


SY .  ,  Jv 

i(o)  Y 

rf  i 


is  of  the  form 


=  b  M  +  b  M  +  +  b  M 

.BY.,  21  1  22  2  2k  k 


(43) 


i(°)  Y 


r^i 


Multiplying  equation  (43)  by  S  and  then  squaring  this  pro- 

i(o)  i (o) 


(42) 


duct,  we  get 


); 


•  . 


; 
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(  ^2  )2  s2 

3Yi(o)  Yr;tl  Yi(o)Yi(o) 


2  2  2  2 
b21Ml  +  b22M2  +  ' 


2  2 
•  +  b2kMk 


+  b21b22MlM2  +  b22b21M2Ml  + 


+  b21b2kMlMk  +  b22b2kM2Mk 


+ 


b2kb22M2Mk 


+  ... 


SY  Y  (44) 


Substituting  equation  (41)  into  equation  (44)  and  summing  over  all 
of  the  observed  Y^' s,  we  get 


which  is  the  exact  equation  from  which  we  can  calculate  the  variance 


of  ^  •  Similarly,  the  variance  of  3^.  is  of  the  same  form  as  j3^ 


and 


32: 


k  k 


Vk 


=  L 


m=l  j=l 


n 


bi  bv  • 
km  kj 


W 


Y.  (  AB3  /  v  Q 
.  ,  i(o)  m  x .  ,Y. ,  N ,3 .  y  J 

i=l  J,  i ( o)  K j/m 


f— i) 


\33 


x .  ,Y.  .  .  ,3  ,  . 
Ji  i(o)  m^j 


(46) 


Now  we  proceed  to  calculate  the  covariances  of  our  constants 


Suppose  we  multiply  the  product  of  equations  (37)  and  (43)  by 

S2 

Y  Y 
i(o)  i(o) 


This  gives  us 


■ 


, 


I 

I 
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\SY. 


)Y.,  */v  \BY.  .  N/v  Y .  .  v  Y.  .  ' 

1(°)  Yr?Li  1(0)  Y^  1(0)  1(0) 


bllb21Ml  +  bllb22MlM2  +  +  bllb2kMlMk 


b12b2kM2Mk 


+  b12b2iM2Ml  +  b12b22M2  +  + 


+  . 


+  b„b„M,M,  +blkb22MkM2+  ...  +blkb2kMk 


lk  21  k  1 


Y  Y 
i(°)  i(o) 


(47) 


and  substituting  equation  (41)  into  equation  (47)  and  then  summing 
over  all  of  the  Y^^'s,  we  get 


t  SP1  V  3P2  \,2 

\SY.  .  J\SY.,  J w  *w  v 

,  l(o)  l(o)  l(o)  l(o) 


Y-'  'Y-'  '  =  V2 


k  k 


=  L 


m=l  j=l 


n 


(^) 


hmhj  ZY,  .  v^.F  .  v  a  .~K .  .  v 

_  1=1  l(o)  1,1  Xj.’Yi(o)’ej^  J  Xj.’Yi(o)’PmA 


(£) 


f 

ri 

I 


(48) 


from  which  we  can  calculate  the  covariance  of  3^32’  Similarly,  the 
covariance  of  3  8  is  of  the  form 

1  q 


Yq 


=  L 


k  k 

I 

m=l  j=l 


n 


bi  b 

lm  qj 


W. 


/SAY. 
i 


,SAY.. 

Y.  ,  v \S3  )  v  a  '33 . 

.  .  i(o)  m  x.  ,Y .  ,  x,3./  i 

i=l  j  .  i ( o)  ’Pj£n  J 


x .  ,Y. f  .  ,3  ,  . 
Ji  i(o)  Wj 


(49) 


where  b  .  is  the  element  of  the  inverse  matrix  of  order  (k,k) . 

qj 

Similarly,  the  covariance  of  393  is  of  the  form 

c.  q 

k  k 


2  2 

S8  B  =  L 

323q 


m=l  j=l 


n 


b  ry  b  . 

2m  qj 


q 

/SAY 


W_ 


^  Y.  ,  AS3  1  v  0  VS3.  /  v  a 

.  .  i (o)  m  x.  ,Y.  .  .  ,3  .  J  x.  ,Y.  .  .,3 
i=l  Ji  i(o)  j?hn  J  Ji  i(o) 


SAY 

l 


(50) 


3 
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Therefore,  we  see  from  equations  (42),  (45),  (46),  (48),  (49),  and 
(50)  that  we  can  write  the  single  equation 


(51) 


where  b  is  the  element  of  the  inverse  matrix  of  order  (k,k) . 
pm 

Equation  (51)  is  the  generalized  exact  equation  from  which  we  can 
calculate  all  variances  and  all  covariances  of  the  constants  eval¬ 
uated  because  if  p  =  q  =  1 ,  then  equation  (51)  is  nothing  but  the 
expression  for  calculating  the  variance  of  3^  and  is  just  equation 
(42).  Now  if  p=l  and  q=2 ,  then  equation  (51)  is  nothing  but  equation 
(48)  and  is  the  expression  for  calculating  the  covariance  of 

Suppose  we  simplify  equation  (51)  by  substituting  equation  (33) 
into  equation  (51): 


(52) 


From  equation  (32)  ,  we  have 


A 


[  ^  .  ]  / 1  i  \ 

jm  (k,k) 


T 


(32) 


Let  the  inverse  of  A,  designated  as  A  ,  be 


A 


-1 


^bqj4k,k)  '-bjq4k,k) 


(53) 


P  . 
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Then 


where 


A_1A  =  I 

^  °mq  ^ (k,k) 


k 

c  =  )  a  .b . 

mq  mj  jq 

j  =  l 


(54) 


(55) 


and  it  follows  that  c  (m=q)  =  1  and  c  (m^q)  =  0.  Therefore,  we 

mq  mq 

see  that  the  first  term  which  appears  on  the  right-hand  side  of 
equation  (52)  is 


a  .b .  j  =  b 
mj  jq/  pq 


(56) 


Therefore,  substituting  equation  (56)  into  equation  (52),  we  get 


k  k 


m=l  j  =  l 


b  b  . 

pm  qj 


n 


w. 


i=l 


i(o) 


2 

5  AY. 
_ _ i 

53  .53 

J 


(57) 


which  is  the  exact  mathematical  equation  from  which  we  can  calculate 
all  variances  and  covariances  of  the  constants  evaluated.  Notice 
that  since  the  coefficient  matrix  is  symmetr  ical ,  then  it  is  only- 
necessary  to  evaluate  half  of  the  elements  of  the  coefficient  matrix 
and  of  the  inverse  matrix.  In  other  words,  we  need  only  evaluate 
those  elements  which  lie  on  or  above  the  principal  diagonal  of  A  and 
of  A  ^  [The  principal  diagonal  is  composed  of  those  elements  which 
have  the  same  row  and  column  index] . 


' 


■ 
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